Skip to content

feat(kms): authenticate RA-TLS clients by attestation instead of by issuer - #1106

Merged
kvinwang merged 8 commits into
nextfrom
feat/ratls-self-signed-client-certs
Aug 24, 2026
Merged

feat(kms): authenticate RA-TLS clients by attestation instead of by issuer#1106
kvinwang merged 8 commits into
nextfrom
feat/ratls-self-signed-client-certs

Conversation

@kvinwang

@kvinwang kvinwang commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

feat(kms): authenticate RA-TLS clients by attestation instead of by issuer

Server-side half of removing the temp-CA workaround. The client-side half — guests
minting self-issued certificates — is split into a separate PR, because it narrows
which KMS versions a guest image can talk to and deserves its own decision.

Problem

Rocket configures mutual TLS through rustls' WebPkiClientVerifier, which pins a CA
(core/lib/src/tls/config.rs:453-467). mandatory = false compiles to
.allow_unauthenticated(), whose semantics are "you may omit a certificate, but if you
send one it must chain to the pinned CA". There is no "accept any certificate, let the
application decide" mode.

An RA-TLS certificate is self-issued and carries its identity in a TEE quote whose
report_data binds the certificate's own SPKI. There is nothing for it to chain to. So
dstack invented something: GetTempCaCert hands every caller the temp CA certificate
and its private key (kms/src/main_service.rs:497-512), the caller mints itself a
certificate from it, and rustls is satisfied.

That CA establishes nothing. The endpoint is unauthenticated, so the key is public by
design — anyone can mint a chain-valid client certificate. The check that has always
carried the meaning runs afterwards, in ra-rpc/src/rocket_helper.rs:529-551:

attestation.into_v1()
    .verify_with_ra_pubkey(&pubkey, &quote_verifier.verifier).await
    .context("invalid quote")?

and the KMS is explicit about this — RpcHandler::construct (main_service.rs:567-571)
drops remote_app_id/remote_app_info and keeps only the verified attestation.

The consequence is that the TLS layer admits everyone, so every authorization check has
to be remembered by hand in each handler. A handler that forgets ensure_attested() is
silently open. This is the "attestation is the trust boundary — not TLS, not the network"
principle from .agent/CODING_TASTE.md, enforced one layer too late.

Reported as #561 and
#619; previously closed as documented.

Fix

Replace the chain check with the check that means something. Rocket master already
exposes tls::Resolver (core/lib/src/tls/resolver.rs), which returns an arbitrary
Arc<rustls::ServerConfig> per ClientHello — so no fork is needed and no rocket API is
being abused.

ra-rpc::ratls_client_verifier adds:

  • RaTlsClientVerifier, a rustls::ClientCertVerifier that advertises no CAs, ignores
    the issuer, and requires the certificate to carry a decodable RA-TLS attestation;
  • RaTlsClientAuth, a tls::Resolver serving one ServerConfig wired to it.

The KMS attaches it with one line. Everything downstream is untouched: rocket populates
peer_certs straight from rustls (listener/connection.rs:88-92, gated only on the
mtls cargo feature, not on [tls.mutual]), and rocket::mtls::Certificate only parses
(mtls/certificate.rs:114-133). rocket_helper.rs and the gateway needed no changes.

The expensive half stays out of the handshake. verify_client_cert is synchronous
while quote verification needs I/O (collateral fetch, auth API). Doing it there would
hand unauthenticated peers a lever to drive that I/O, so the verifier does only local
work and rocket_helper.rs + the service handlers keep doing the rest exactly as before.

No client changes required

Guests and KMS↔KMS onboarding still fetch the temp CA and mint their client certificates
from it. Those certificates are now accepted for the attestation they carry rather than
for their issuer, so every existing client keeps working unchanged, in both directions:
old guests against a new KMS, and this KMS against everything that talks to it today.

What changes:

  • the TLS layer went from admitting any certificate signed by a publicly-known key to
    requiring an attested one — a handler that forgets ensure_attested() is no longer
    silently open;
  • a self-issued certificate is now also accepted, which is what makes it possible to
    migrate callers off GetTempCaCert later.

GetTempCaCert and the temp CA are untouched and still needed. Both remaining callers
are annotated in kms_rpc.proto and on the handler, so the next reader does not have to
re-derive why a CA private key is being served.

[rpc.tls.mutual] is no longer the trust anchor and is dropped from kms.toml and the
KMS config templates (tools/dev-stack.sh, kms/dstack-app/compose-*.yaml,
crates/dstack-cli-core/src/config.rs, test-suites/, the KMS tutorials). Leaving the
section in an existing deployment's config is inert, not an error. The gateway's
[tls.mutual] is deliberately untouched: it pins the KMS root CA, which is a real trust
anchor, since the app identity in those certificates is asserted by the KMS at issuance.

Verification

Two KMS binaries — unmodified origin/next and patched — same dev config otherwise,
probed with curl. Client certificates carry a syntactically valid but deliberately
non-binding attestation, so a reply of invalid quote: report data mismatch is the app
layer proving it still verifies quotes after the handshake succeeded.

client certificate control (origin/next, pins tmp-ca) patched (no [rpc.tls.mutual])
minted from that KMS's temp CA handshake OK → invalid quote: report data mismatch handshake OK → invalid quote: report data mismatch
self-issued, with attestation handshake REJECTED handshake OK → invalid quote: report data mismatch
self-issued, no attestation handshake REJECTED (TLS alert, handshake failure (552))
none App not allowed: No attestation provided
none → GetMeta returns ca_cert
none → GetTempCaCert still served

Row 1 is the compatibility guarantee: what every client sends today is accepted by both.
Row 2 is the new capability. Row 3 shows the verifier is enforcing, not just permitting.
Rows 4–6 show the unauthenticated surface and the app-layer gate are unchanged.

The patched KMS starts with no [rpc.tls.mutual] section at all and logs
RA-TLS client certificate verification enabled. The same matrix was re-run against a
KMS booted from a config generated by the cleaned tools/dev-stack.sh template, to
confirm the config edits produce a working KMS and not just a parsing one.

KMS↔KMS onboarding was checked by reading rather than executed (it needs two KMS CVMs):
gen_ra_cert (onboard_service.rs:747-767) produces a CA-signed certificate carrying an
attestation, which is exactly row 1; the onboard listener itself never merges rpc into
its figment, so it has no tls section, serves plain HTTP, and the fairing is not
attached to it. Nothing in kms/src/ reads mutual.

Automated coverage:

  • ra-rpc/src/ratls_client_verifier.rs — 4 unit tests: CA-signed accepted, self-signed
    accepted, no-attestation rejected, expired rejected (the validity window webpki used to
    enforce is now enforced by the verifier).
  • ra-rpc/tests/ratls_client_auth.rs — end-to-end through a real Rocket server with the
    fairing attached, asserting the same four outcomes at the TLS layer and that
    rocket::mtls::Certificate still receives the certificate.

cargo fmt --check, cargo clippy -- -D warnings -D clippy::expect_used -D clippy::unwrap_used --allow unused_variables, and the touched crates' test suites are
clean.

Follow-ups

  • Guests minting self-issued certificates, which lets them stop calling GetTempCaCert.
    Split out because it narrows KMS compatibility for a guest image.
  • KMS↔KMS onboarding likewise; once both are migrated, GetTempCaCert and the temp CA
    can be deleted.
  • The gateway's register_cvm prefers the certificate extension over the verified
    attestation (gateway/src/main_service.rs:1529-1538), which is sound only because it
    pins a real CA. Making that dependency explicit is worth its own change.

Copilot AI lite review requested due to automatic review settings August 23, 2026 15:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@kvinwang
kvinwang force-pushed the feat/ratls-self-signed-client-certs branch from 5b9521b to 8f51f71 Compare August 23, 2026 15:13
Comment thread dstack/ra-rpc/tests/ratls_client_auth.rs Dismissed
@kvinwang
kvinwang merged commit f6bc592 into next Aug 24, 2026
15 checks passed
@kvinwang
kvinwang deleted the feat/ratls-self-signed-client-certs branch August 24, 2026 02:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants